Combining Front Vehicle Detection with 3D Pose Estimation for a Better Driver Assistance

Driver assistant systems enhance traffic safety and efficiency. The accurate 3D pose of a front vehicle can help a driver to make the right decision on the road. We propose a novel real‐time system to estimate the 3D pose of the front vehicle. This system consists of two parallel threads: vehicle rear tracking and mapping. The vehicle rear is first identified in the video captured by an onboard camera, after license plate localization and foreground extraction. The 3D pose estimation technique is then employed with respect to the extracted vehicle rear. Most current 3D pose estimation techniques need prior models or a stereo initialization with user cooperation. It is extremely difficult to obtain prior models due to the varying appearance of vehicles’ rears. Moreover, it is unsafe to ask for drivers’ cooperation when a vehicle is running. In our system, two initial keyframes for stereo algorithms are automatically extracted by vehicle rear detection and tracking. Map points are defined as a collection of point features extracted from the vehicle’s rear with their 3D information. These map points are inferences that relate the 2D features detected in following vehicles’ rears with the 3D world. The relative 3D pose of the onboard camera to the front vehicle rear is then estimated through matching the map points with point features detected on the front vehicle rear. We demonstrate the capabilities of our system by testing on real‐time and synthesized videos. In order to make the experimental analysis visible, we demonstrated an estimated 3D pose through augmented reality, which needs accurate and real‐time 3D pose estimation.